                          ===========
                            README
                          ===========
                                   

		QLogic iSCSI Boot Software Initiator
                  Copyright (c) 2015 QLogic Corporation
                                 All rights reserved.  
                              

******************************** IMPORTANT NOTE *******************************
*                                                                             *
* PXE-BASED iSCSI BOOT IS BETA ONLY AND IT WILL NOT BE PRODUCTIZED IN THE     *
* FUTURE.                                                                     *
*                                                                             *
*******************************************************************************

1. INTRODUCTION
===============

This file describes the QLogic implementation of iSCSI Boot software
initiator.  QLogic iSCSI Boot software initiator is implemented in 2 
forms:

 * A Network Bootstrap Program (NBP) which is downloaded to the host through 
   PXE boot. 

 * A ROM binary which is programmed on the device's NVRAM.

In the first implementation, iSCSI DOS initiator is downloaded to the host 
through PXE boot. PXE code then jumps to iSCSI Boot software for execution. 
During iSCSI Boot software's initialization, it removes Base Code from the 
memory to minimize usage of conventional memory and then binds with UNDI 
driver to form an iSCSI stack.

In the second implementation, iSCSI ROM driver is stored in NVRAM.  iSCSI ROM
driver image is loaded to host memory by the Multiple Boot Agent (MBA).
During a network boot, MBA ROM loads UNDI and iSCSI ROM driver to the 
host conventional memory and forms an iSCSI stack as in the first 
implementation. In this implementation, there is no requirement of PXE 
server.

The iSCSI stack for both cases is shown as follows:

INT 13H system handler
----------------------
iSCSI Boot software
---------------------
TCP/IP stack
---------------------
UNDI Driver
---------------------

iSCSI boot software hooks system INT 13H handler so that it can redirect
disk access requests transparently from a real-mode operation system to 
remote disk via iSCSI protocol. 

Since DOS is purely a real-mode operating system, it continues to use INT 13H 
calls to access the disks after the DOS operating is loaded. For instance, if a 
DOS application is executed or files are copied from the iSCSI drives, DOS will 
invoke series of INT 13H calls to access iSCSI disks. 

In Linux O/S boot, boot loaders such as GRUB (Grand Unified Boot Loader) or 
LILO (Linux Loader) uses INT13H service calls to load an initial RAM disk 
(initrd).  This initial RAM disk image contains configuration information, 
the necessary kernel loadable modules for the NIC and iSCSI initiator, and some 
managing applications.  Once Initial RAM disk is read over and run, it executes 
initrd's startup script (linuxrc).  This startup script does the following:
 
  1. Loads the kernel modules (SCSI driver,network driver, iSCSI driver, etc...) 
  2. Run iSCSI querying application to retrieve iSCSI information from the 
     network device which was stored there by iSCSI Boot driver and UNDI.
  3. Use iSCSI information to configure network interface such as IP address,  
     subnet mask, etc... 
  4. Use iSCSI information to configure UNH iSCSI initiator.
  5. Switch root disk to iSCSI target.

At this point, iSCSI boot initiator is no longer used.  The native Linux stack 
is simply depicted as follows:

Linux file system/Linux kernel
-----------------------------
Linux SCSI Middle Module
-----------------------------
UNH iSCSI Initiator
-----------------------------
Linux TCP/IP stack 
-----------------------------
L2 Linux driver (e.g tg3.o, bnx2.o) 
----------------------------

2. DRIVER COMPONENTS
=====================
The following components are released:

release.txt : documents revision history of iSCSI driver 
readme.txt  : describes overall iSCSI functionality and installation procedure.
ibootvMMnn.bb  : This file contains various 4 software modules which are designed to
              get programmed on the device's NVRAM.  
  
              * iSCSI DOS Initiator driver
              * Default iSCSI configuration block.

              One or more components can be programmed on the device's NVRAM by 
              using command "nvm upgrade" in XDIAG shell or command "iscsiprg" 
              in B57DIAG shell.
bibt-*.noarch.rpm: RPM package containing iSCSI Boot tools used to create the 
                   customized INITRD image capable of iSCSI Boot. The following 
                   tools and their associated man pages are included: 
                   iscsi_setup.sh and query_iscsi.  Installing this package will
                   place a folder of the same name into the /opt/bcm directory,
                   which contains the tools.


3. iSCSI PARAMETERS
===================
iSCSI parameters consist of the following:

* TCP/IP parameters: IP address, subnet mask, and default gateway of the 
  iSCSI boot client.

* iSCSI initiator parameters: iSCSI name, CHAP ID, and CHAP password 
  (aka CHAP secret).

* iSCSI target parameters: iSCSI target name, IP address, TCP port number,
  Boot LUN, CHAP ID, and CHAP password.

* Secondary Device parameters(Multipath I/O configuration): MAC Address,
  Use Independent Target Portal.

These parameters can be stored on the device's NVRAM or dynamically 
acquired via DHCP.

3.1 Acquisition of parameters via DHCP 

iSCSI boot software acquires client's IP address information, client's 
host name, client's Initiator information, and iSCSI Target Information 
from the DHCP server. iSCSI Target information is defined as one of the 
DHCP options called Root Path (option 17) or encapsulated in DHCP option 43.  
This option is encoded as follows per RFC 4173 (Bootstrapping Clients using
iSCSI protocol)

"iscsi":"<server name>":"<protocol>":"<port>":"<LUN>":"<targetname>

For instance,
iSCSI target with IP address 60.2.1.2 and iSCSI target name is 
"iqn.2002-10.edu.unh.iol.iscsi.draft20-target".

Root path should be created as 
"iscsi:60.2.1.2:tcp:3260:0:iqn.2002-10.edu.unh.iol.iscsi.draft20-target"

In addition to this standard DHCP option, QLogic iSCSI boot initiator also 
supports a proprietory option which is encapsulated within DHCP option 43.  
This option can include iSCSI initiator name, one or two iSCSI target
information.  The format of the suboptions in the DHCP option 43 are in 
TLV - <TAG> <LEN> <VALUE> format. All the supported suboptions are described as 
follows:

        Option 201: First iSCSI Target Information which is in standard 
        format as root-path.
        "iscsi":"<server name>":"<protocol>":"<port>":"<LUN>":"<targetname>

        Option 202: Second iSCSI Target Information which is in standard 
        format as root-path.
        "iscsi":"<server name>":"<protocol>":"<port>":"<LUN>":"<targetname>

        Option 203: iSCSI Initiator Name.

3.2 Storage of Configuration in NVRAM

Complete or partial iSCSI parameters can be stored in NVRAM. Users can configure 
iSCSI parameters with Comprehensive Configuration Management tool.

3.3 Multipath I/O configuration settings

A one-to-one target correlation between Primary and Secondary devices should
be used when enabling "Use Independent Target Portal" parameter.
For example, if the Primary device configuration has Target1 connection
enabled, and Target2 connection disabled; the Secondary device should also
have Target1 connection enabled, and Target2 connection disabled.

4. INSTALLATION
================

There are a number of services and steps that need to be taken to set up an iSCSI 
network.

4.1. Setup Initiator 

As mentioned in the previous section, iSCSI DOS initiator driver can be downloadable 
via PXE or can be stored in the device's NVRAM.  This section will describe both 
methods:

4.1.1 iSCSI driver stored on PXE server

  - Setup a PXE server such as Intel PDK.  Install iSCSI boot software iscsi.0
    as a bootable image.

  - Setup DHCP server with the following information per client's MAC address:
     * Client's IP address 
     * Client's host name
     * Root Path 

4.1.2 iSCSI driver in NVRAM 

Users can program one or more iSCSI components on the device's NVRAM by using 
XDIAG or B57DIAG.

For Teton devices:
    * To program iSCSI initiator 
            nvm upgrade -ib a:/ibootvMMnn.bb 

      Note that if iSCSI configuration parameter block doesn't exist on the 
      NVRAM, this command also program default iSCSI configuration block 
      as well.
     
For BCM570x devices:
    * To program iSCSI initiator 
           iscsiprg -fa:\ibootvMMnn.bb 

      Note that if iSCSI configuration parameter block doesn't exist on the 
      NVRAM, this command also program default iSCSI configuration block 
      as well.
     
4.2. SETUP TARGET DISK

4.2.1 Linux partition

Creating a bootable Linux disk image on the iSCSI target involves three major steps:
   * Create DISK image from the clean install.
   * Customize initial RAM disk. 
   * Customize Boot Loader to load customized initrd.img.

4.2.1.1 Creation of Disk Image from a Clean Install

  - Install Red Hat Enterprise Linux 4 Update 5 (or greater) OR Red Hat Enterprise Linux 5 OR SuSE SLES 9 SP3 (or greater) OR SuSE SLES 10 SP1 (or greater) on a system with identical hardware 
    configuration of the iSCSI clients. 

  - Configure network interface not to be shutdown during system shutdown or 
    reboot.

        RedHat: chkconfig --level 0123456 network on

                                  -OR-

        SuSE  : Use Yast configuration tool to change Network runlevel services
                      

  - Install iSCSI initiator software.  The linux-iscsi initiator software package is included in the Red Hat 4/SuSE 9 SLES distribution media.  The open-iscsi initiator package is included in the Red Hat 5/SuSE 10 SLES SP1 distribution media.  The QLogic iSCSI Offload driver is also packaged with a modified version of open-iscsi.


  - Access iSCSI target via iSCSI initiator software. See documentation that comes with
    iSCSI software package for more usage details.

Note:  If CHAP authentication is to be used in conjunction with the iSCSI offload driver on Red Hat Enterprise Linux 5, it will be necessary to modify the /etc/iscsi/iscsid.conf file prior to copying the contents of the local disk to the target drive.  Edit this file such that CHAP authentication is enabled by default.

  - Copy the entire disk from local drive to iSCSI target drive sector-by-sector. 
         dd if=/local_disk of=/target_disk bs=8192

  - Restart the iSCSI service
     
         /etc/init.d/iscsi restart

Note:  If the iSCSI offload path is used, it will be necessary to modify one of two scripts on the newly created disk image.  The exact file will depend on the disk image being used.
       If the iSCSI non-offload path is used, these scripts will not need to be modified, and the user can proceed to the next section Customize the Initial RAM Disk.

  - Mount the / partition of the newly created image.  Assume that the root 
    partition of the iSCSI target drive is on /dev/sda2.

         mount /dev/sda2 /mnt

  - Use a text editor to modify one of the two following files based on the
    OS distribution being used.

  RH5: /mnt/etc/init.d/iscsid

    Comment out or remove the following lines:
      
      chkconfig --level 06 network off
      rm /etc/rc0.d/*network
      rm /etc/rc6.d/*network


  SuSE 10: /mnt/etc/init.d/boot.open-iscsi

    Comment out or remove the following lines:

      if ! grep -q iscsi_tcp /proc/modules ; then
         rc_failed 6
         rc_exit
      fi

  - Save changes to the file and unmount the partition

        sync
        umount /mnt


4.2.1.2 Customizing Initial RAM Disk (INITRD)
=============================================
The iscsi_setup.sh script, part of this release, will build and customize an
INITRD image (iscsi-initrd.img) with the necessary support modules to initialize
a network interface and create an iSCSI connection.

  - Execute the iscsi_setup.sh script.  It may be necessary to run dos2unix
    on this script, to ensure that it is formatted properly.
	
	./iscsi_setup.sh

Note:  The iscsi_setup.sh script will determine which network driver(s) to 
include in the image, either TG3 or BNX2 or both, based on a query to 'lsmod'. 
Ensure that all necessary support modules have been loaded prior to executing
this script.  A temporary directory (/mnt/iscsiboot) will be created/modified
when creating the new image.  If such a directory already exists, please
backup the original contents or rename it, as the script will remove this 
directory when it has completed.


4.2.1.3 Customizing Boot Loader 
===============================

First of all, you have to copy customized initrd to the boot partition on the
iSCSI target disk.  Assume that the boot partition of the iSCSI target is on
/dev/sda1.

   - mkdir /mnt/myboot
   - mount /dev/sda1 /mnt/myboot
   - cp iscsi-initrd.img /mnt/myboot
   
Next, you have to edit boot loader configuration file to load customized
initrd. Use a text editor to modify the following line in
/mnt/myboot/grub/menu.lst

     initrd /iscsi-initrd.img
 

5. LIMITATION
=============
1. This release doesn't support DNS. Therefore iSCSI Target has to be 
   given as x.x.x.x.


6. APPENDIX
============

6.1. Improvement of iSCSI WRITE operations

iSCSI DOS initiator has been optimized for iSCSI Read operations.  In order to 
keep the memory footprint small, TCP/IP stack has been implemented so that it 
doesn't maintain a large retransmission buffer. It only sends out the next TCP 
packet with payload until it receives a TCP ACK for the previously sent packet. 
Most of the operating systems use TCP ACK delay mechanism to reduce ACK packets
being sent out.  In other words, after receiving a TCP packet, the TCP/IP will
not send out TCP ACK after a timout period (e.g 200ms in Windows enviromnent).  

In order to improve iSCSI Write operations, TCP ACK delay mechanism should be 
disabled on the iSCSI target.  In Windows environment, this mechanism can be 
disabled by creating a TcpAckFrequency register and setting this value to 1. 
 See the Microsoft
documentation for details:

      http://support.microsoft.com/default.aspx?scid=kb;en-us;328890

6.2. Description of iSCSI_SETUP Script

This script is called from linuxrc (kernel 2.4) or init (kernel 2.6) startup. 
This script brings up the ethernet interface.  It then uses query_iscsi 
utility to retrieve iSCSI information from the interface.  It then uses these 
parameters to configure IP adress of the ethernet interface and iSCSI initiator.

#!/bin/ksh

echo "Querying iSCSI information..."
HOST_NUM=0
TARGET_NAME=`query_iscsi TARGET_NAME`
TARGET_IP_ADDRESS=`query_iscsi TARGET_IP_ADDRESS`
TARGET_TCP_PORT=`query_iscsi TARGET_TCP_PORT`
INITIATOR_NAME=`query_iscsi INITIATOR_NAME`
CLIENT_IP_ADDRESS=`query_iscsi CLIENT_IP_ADDRESS`
DEFAULT_GATEWAY=`query_iscsi DEFAULT_GATEWAY`
SUBNET_MASK=`query_iscsi SUBNET_MASK`
AUTHEN_MODE=`query_iscsi AUTHEN_MODE`
INITIATOR_ID=`query_iscsi INITIATOR_ID`
INITIATOR_PW=`query_iscsi INITIATOR_PW`
TARGET_ID=`query_iscsi TARGET_ID`
TARGET_PW=`query_iscsi TARGET_PW`
MAC_ADDRESS=`query_iscsi MAC_ADDRESS`

echo "Target Name = " $TARGET_NAME
echo "Target IP Address = " $TARGET_IP_ADDRESS
echo "Target TCP Port = " $TARGET_TCP_PORT
echo "Initiator Name = " $INITIATOR_NAME
echo "Client IP Address = " $CLIENT_IP_ADDRESS
echo "Default Gateway = " $DEFAULT_GATEWAY
echo "Subnet Mask = " $SUBNET_MASK
echo "Authen Method = " $AUTHEN_MODE
echo "Initiator ID = " $INITIATOR_ID
echo "Target ID = " $TARGET_ID
echo "MAC Address = " $MAC_ADDRESS

echo "Loading tg3.ko module"
insmod /lib/tg3.ko

# Determine iSCSI Boot ethernet interface
INTERFACE=`ifconfig -a | grep $MAC_ADDRESS | awk '{print $1}'`

ifconfig $INTERFACE

ifconfig $INTERFACE $CLIENT_IP_ADDRESS netmask $SUBNET_MASK up

# Give adapter a chance to negotiate the link
sleep 5

# Use one of the following three sections based on the initiator being utilized.

####################################################################
#                                                                  #
#  The following section is for use with the Linux-iscsi Initiator #
#                                                                  #
####################################################################

echo "InitiatorName=$INITIATOR_NAME" > /etc/initiatorname.iscsi
echo "DiscoveryAddress=$TARGET_IP_ADDRESS" > /etc/iscsi.conf
#If CHAP authentication is enabled, uncomment the following 2 lines
# echo "  OutgoingUsername=$INITIATOR_ID" >> /etc/iscsi.conf
# echo "  OutgoingPassword=$INITIATOR_PW" >> /etc/iscsi.conf
echo "Enabled=No" >> /etc/iscsi.conf
echo "TargetName=$TARGET_NAME" >> /etc/iscsi.conf
echo "  Enabled=Yes" >> /etc/iscsi.conf

insmod /lib/libcrc32c.ko
insmod /lib/crc32c.ko
insmod /lib/md5.ko
insmod /lib/scsi_transport_iscsi.ko
insmod /lib/iscsi_sfnet.ko
sleep 2

if [ -e "/dev/iscsictl" ]; then
  rm /dev/iscsictl
fi

# Following code snippet taken from '/etc/init.d/iscsi' to start linux-iscsi daemon.
if [ ! -f /dev/iscsictl ]; then
        while read major device 
        do
        if [ "$device" == "iscsictl" ]; then
          mknod /dev/$device c $major 0
        fi
        done < /proc/devices
fi

echo -n "Starting iscsid: "
/sbin/iscsid -f /etc/iscsi.conf

# Allow iscsid time to complete startup
sleep 2

#########################################
#  End of Linux-iscsi Initiator section #
#########################################

####################################################################
#                                                                  #
#  The following section is for use with the Open-iscsi Initiator  #
#                                                                  #
####################################################################

echo "Starting iscsid"
/sbin/startproc /sbin/iscsid -c /etc/iscsid.conf

# The following 4 lines will search for a previous db record with $TARGET_NAME and delete if found.  If the db that resides on INITRD does not contain any records with $TARGET_NAME, then the following lines may be omitted

PREVIOUS=`iscsiadm -m node show | grep $TARGET_NAME | awk '{print $1}' | awk '{gsub(/\[|\]/, ""); print}'`
if [ $PREVIOUS ]; then
  iscsiadm -m node -r $PREVIOUS -o delete
fi

RECORD=`iscsiadm -m node --op new --portal $TARGET_IP_ADDRESS | awk '{field = $NF }; END { print field } ' | awk '{gsub(/\[|\]/, ""); print}'`

iscsiadm -m node --record $RECORD --op update -n node.name -v $TARGET_NAME
# If CHAP authentication is enabled, uncomment the following 3 lines
#iscsiadm -m node --record $RECORD --op update -n node.session.auth.authmethod -v $AUTHEN_MODE
#iscsiadm -m node --record $RECORD --op update -n node.session.auth.username -v $INITIATOR_ID
#iscsiadm -m node --record $RECORD --op update -n node.session.auth.password -v $INITIATOR_PW
iscsiadm -m node --record $RECORD --login

#########################################
#  End of Open-iscsi Initiator section  #
#########################################

####################################################################
#                                                                  #
#  The following section is for use with the UNH Initiator         #
#                                                                  #
####################################################################



echo "Running insmod /lib/unh_iscsi_initiator.ko"

insmod /lib/unh_iscsi_initiator.ko

iscsi_manage init restore host=$HOST_NUM target=0
iscsi_manage init set host=$HOST_NUM target=0 InitiatorName=$INITIATOR_NAME
iscsi_manage init set host=$HOST_NUM target=0 TargetName=$TARGET_NAME
iscsi_manage init set host=$HOST_NUM target=0 AuthMethod=$AUTHEN_MODE
iscsi_manage init set host=$HOST_NUM target=0 SessionType=Normal 
#If CHAP authentication is enabled, comment out the following line
# iscsi_manage init force s cl=256 host=$HOST_NUM target=0 pn="$INITIATOR_ID" px="$INITIATOR_PW"

iscsi_config up ip=$TARGET_IP_ADDRESS host=$HOST_NUM target=0 port=$TARGET_TCP_PORT

#########################################
#  End of UNH Initiator section         #
#########################################


7. COMPATIBILITY
=================

The following iSCSI targets have been tested with iSCSI Boot agent:

Cisco MDS 9216
Cisco SN 9420
Equallogic 100E
Stringbean Software Target
Dell/EMC AX100i
SanRad V Switch 3000
UNH Target

8. KNOWN ISSUES
===============

8.1 UNH iSCSI Initiator

UNH iSCSI initiator installs notifying handler in the event of shutdown
or reboot.  The UNH handler cleans up the resource once it's invoked.  
During shutdown, SCSI middle driver (sd_mod.ko) invokes iSCSI driver 
(e.g UNH iSCSI Initiator) to send out last disk sync command to the 
target if WCE (Write Cache Enable) is enabled on the target.  This can
cause UNH to display fatal errors ("queuecommand called after shutdown")
since UNH already clears out its resource.  To workaround this issue,
WCE should be enabled on target if it's enabled.

8.2 Open-iscsi Initiator

Connection issue with Stringbean target.  During system boot, the following error is encountered:
iscsi: can not unicast skb
iscsi: detected conn error

To workaround this issue, download/use latest open-iscsi initiator version at http://www.open-iscsi.org.
 
8.3 MS Initiator v2.0.3
Problem:
  In an environment with multiple iSCSI Boot capable NICs, the system is unable
  to successfully boot from the primary(*) adapter after any secondary(#) 
  adapter is used to boot up the system.
Solution: 
  Edit system registry key: 

   System\CurrentControlSet\Services\Tcpip\Parameters\DisableDHCPMediaSense

  Set value to 0.  Doing this will enable the media sense to attempt to           detect a link on the primary(*) NIC.

*:  NIC used during the imaging of the local filesystem to the target.
#:  All others excluding the primary.

8.4 Dell/EMC AX100i
Enable TCP timestamp parameter when connecting to this target. 

8.5 Locally Assigned Address
In a Windows environment, if an iSCSI Boot interface is configured with a Locally Assigned Address, Windows will fail to assign the iBFT-obtained IP address to that interface.
LAA is not supported with iSCSI Boot.

8.6 Installing RPM on RH6.0
Use --nodeps flag during RPM installation on RH6.0.
i.e. rpm -ivh --nodeps bibt*-.rpm

8.7 Direct iSCSI target installation on Red Hat 6.1 - Switching IP versions.
If attempting to boot using IPv6 on a direct target installed copy of Red 
Hat 6.1, it will be necessary to run the iscsi_setup.sh script to create a new 
boot INITRD.  The native 'iscsistart' utility used by the O/S installer, does 
not have IPv6 support.  It will also be necessary to remove references in the 
boot loader configuration file relating to "iscsi_firmware ip=ibft ifname=...".
These keywords are using by 'iscsistart' and will cause a failure since it 
cannot resolve the IPv6 addresses in the iBFT successfully.
